Distribution Cache for Graph Data

ABSTRACT

In one embodiment, a system includes a database; and a cache layer comprising one or more cache nodes, the one or more cache nodes operative to: maintain in a memory one or more data structures storing association information describing associations between nodes in a graph a plurality of distributed cache clusters for storing information in the form of a graph, the graph comprising a plurality of nodes, each uniquely identified by a node identifier, and edge information indicating associations between nodes; respond to queries for associations between nodes in the graph by accessing the memory; and forward other queries to the database for processing.

PRIORITY

This application is a continuation under 35 U.S.C. §120 of U.S. patentapplication Ser. No. 13/227381, filed 7 Sep. 2011, which claims thebenefit under 35 U.S.C. §119(e) of U.S. Provisional Patent ApplicationNo. 61/428799, filed 30 Dec. 2010, each of which is incorporated hereinby reference.

TECHNICAL FIELD

The present disclosure relates generally to storing and serving graphdata, and more particularly, to storing and serving graph data with adistributed cache system.

BACKGROUND

Computer users are able to access and share vast amounts of informationthrough various local and wide area computer networks includingproprietary networks as well as public networks such as the Internet.Typically, a web browser installed on a user's computing devicefacilitates access to and interaction with information located atvarious network servers identified by, for example, associated uniformresource locators (URLs). Conventional approaches to enable sharing ofuser-generated content include various information sharing technologiesor platforms such as social networking websites. Such websites mayinclude, be linked with, or provide a platform for applications enablingusers to view web pages created or customized by other users wherevisibility and interaction with such pages by other users is governed bysome characteristic set of rules.

Such social networking information, and most information in general, istypically stored in relational databases. Generally, a relationaldatabase is a collection of relations (frequently referred to astables). Relational databases use a set of mathematical terms, which mayuse Structured Query Language (SQL) database terminology. For example, arelation may be defined as a set of tuples that have the sameattributes. A tuple usually represents an object and information aboutthat object. A relation is usually described as a table, which isorganized into rows and columns. Generally, all the data referenced byan attribute are in the same domain and conform to the same constraints.

The relational model specifies that the tuples of a relation have nospecific order and that the tuples, in turn, impose no order on theattributes. Applications access data by specifying queries, which useoperations to identify tuples, identify attributes, and to combinerelations. Relations can be modified and new tuples can supply explicitvalues or be derived from a query. Similarly, queries identify maytuples for updating or deleting. It is necessary for each tuple of arelation to be uniquely identifiable by some combination (one or more)of its attribute values. This combination is referred to as the primarykey. In a relational database, all data are stored and accessed viarelations. Relations that store data are typically implemented with orreferred to as tables.

Relational databases, as implemented in relational database managementsystems, have become a predominant choice for the storage of informationin databases used for, for example, financial records, manufacturing andlogistical information, personnel data, and other applications. Ascomputer power has increased, the inefficiencies of relationaldatabases, which made them impractical in earlier times, have beenoutweighed by their ease of use for conventional applications. The threeleading open source implementations are MySQL, PostgreSQL, and SQLite.MySQL is a relational database management system (RDBMS) that runs as aserver providing multi-user access to a number of databases. The “M” inthe acronym of the popular LAMP software stack refers to MySQL. Itspopularity for use with web applications is closely tied to thepopularity of PHP (the “P” in LAMP). Several high-traffic web sites useMySQL for data storage and logging of user data.

As communicating with relational databases is often a speed bottleneck,many networks utilize caching systems to serve particular informationqueries. For example, Memcached is a general-purpose distributed memorycaching system. It is often used to speed up dynamic database-drivenwebsites by caching data and objects in RAM to reduce the number oftimes an external data source (such as a database or API) must be read.Memcached's APIs provide a giant hash table distributed across multiplemachines. When the table is full, subsequent inserts cause older data tobe purged in least recently used (LRU) order. Applications usingMemcached typically layer requests and additions into core beforefalling back on a slower backing store, such as a database.

The Memcached system uses a client-server architecture. The serversmaintain a key-value associative array; the clients populate this arrayand query it. Clients use client side libraries to contact the servers.Typically, each client knows all servers and the servers do notcommunicate with each other. If a client wishes to set or read the valuecorresponding to a certain key, the client's library first computes ahash of the key to determine the server that will be used. The clientthen contacts that server. The server will compute a second hash of thekey to determine where to store or read the corresponding value.Typically, the servers keep the values in RAM; if a server runs out ofRAM, it discards the oldest values. Therefore, clients must treatMemcached as a transitory cache; they cannot assume that data stored inMemcached is still there when they need it.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example caching system architecture according toone implementation of the invention.

FIG. 2 illustrates an example computer system architecture.

FIG. 3 provides an example network environment.

FIG. 4 shows a flowchart illustrating an example method for adding a newassociation to a graph.

FIG. 5 is a schematic diagram illustrating an example message flowbetween various components of a caching system.

FIG. 6 shows a flowchart illustrating an example method for processingchanges to graph data.

FIG. 7 is a schematic diagram illustrating an example message flowbetween various components of a caching system.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Particular embodiments relate to a distributed caching system forstoring and serving information modeled as a graph that includes nodesand edges that define associations or relationships between nodes thatthe edges connect in the graph. In particular embodiments, the graph is,or includes, a social graph, and the distributed caching system is partof a larger networking system, infrastructure, or platform that enablesan integrated social network environment. In the present disclosure, thesocial network environment may be described in terms of a social graphincluding social graph information. In fact, particular embodiments ofthe present disclosure rely on, exploit, or make use of the fact thatmost or all of the data stored by or for the social network environmentcan be represented as a social graph. Particular embodiments provide acost-effective infrastructure that can efficiently, intelligently, andsuccessfully scale with the exponentially increasing number of users ofthe social network environment such as that described herein.

In particular embodiments, the distributed caching system and backendinfrastructure described herein provides one or more of: low latency atscale, a lower cost per request, an easy to use framework fordevelopers, an infrastructure that supports multi-master, aninfrastructure that provides access to stored data to clients written inlanguages other than Hypertext Preprocessor (PHP), an infrastructurethat enables combined queries involving both associations (edges) andobjects (nodes) of a social graph as described by way of example herein,and an infrastructure that enables different persistent data stores tobe used for different types of data. Furthermore, particular embodimentsprovide one or more of: an infrastructure that enables a cleanseparation of the data access API from thecaching+persistence+replication infrastructure, an infrastructure thatsupports write-through/read-through caching, an infrastructure thatmoves computations closer to the data, an infrastructure that enablestransparent migration to different storage schemas and back ends, and aninfrastructure that improves the efficiency of data object access.

Additionally, as used herein, “or” may imply “and” as well as “or;” thatis, “or” does not necessarily preclude “and,” unless explicitly statedor implicitly implied.

Particular embodiments may operate in a wide area network environment,such as the Internet, including multiple network addressable systems.FIG. 3 illustrates an example network environment, in which variousexample embodiments may operate. Network cloud 60 generally representsone or more interconnected networks, over which the systems and hostsdescribed herein can communicate. Network cloud 60 may includepacket-based wide area networks (such as the Internet), privatenetworks, wireless networks, satellite networks, cellular networks,paging networks, and the like. As FIG. 3 illustrates, particularembodiments may operate in a network environment comprising socialnetworking system 20 and one or more client devices 30. Client devices30 are operably connected to the network environment via a networkservice provider, a wireless carrier, or any other suitable means.

In one example embodiment, social networking system 20 comprisescomputing systems that allow users to communicate or otherwise interactwith each other and access content, such as user profiles, as describedherein. Social networking system 20 is a network addressable systemthat, in various example embodiments, comprises one or more physicalservers 22 and data store 24. The one or more physical servers 22 areoperably connected to computer network 60 via, by way of example, a setof routers and/or networking switches 26. In an example embodiment, thefunctionality hosted by the one or more physical servers 22 may includeweb or HTTP servers, FTP servers, as well as, without limitation, webpages and applications implemented using Common Gateway Interface (CGI)script, PHP Hyper-text Preprocessor (PHP), Active Server Pages (ASP),Hyper Text Markup Language (HTML), Extensible Markup Language (XML),Java, JavaScript, Asynchronous JavaScript and XML (AJAX), and the like.

Physical servers 22 may host functionality directed to the operations ofsocial networking system 20. By way of example, social networking system20 may host a website that allows one or more users, at one or moreclient devices 30, to view and post information, as well as communicatewith one another via the website. Hereinafter servers 22 may be referredto as server 22, although server 22 may include numerous servershosting, for example, social networking system 20, as well as othercontent distribution servers, data stores, and databases. Data store 24may store content and data relating to, and enabling, operation of thesocial networking system as digital data objects. A data object, inparticular implementations, is an item of digital information typicallystored or embodied in a data file, database or record. Content objectsmay take many forms, including: text (e.g., ASCII, SGML, HTML), images(e.g., jpeg, tif and gif), graphics (vector-based or bitmap), audio,video (e.g., mpeg), or other multimedia, and combinations thereof.Content object data may also include executable code objects (e.g.,games executable within a browser window or frame), podcasts, etc.Logically, data store 24 corresponds to one or more of a variety ofseparate and integrated databases, such as relational databases andobject-oriented databases, that maintain information as an integratedcollection of logically related records or files stored on one or morephysical systems. Structurally, data store 24 may generally include oneor more of a large class of data storage and management systems. Inparticular embodiments, data store 24 may be implemented by any suitablephysical system(s) including components, such as one or more databaseservers, mass storage media, media library systems, storage areanetworks, data storage clouds, and the like. In one example embodiment,data store 24 includes one or more servers, databases (e.g., MySQL),and/or data warehouses.

Data store 24 may include data associated with different socialnetworking system 20 users and/or client devices 30. In particularembodiments, the social networking system 20 maintains a user profilefor each user of the system 20. User profiles include data that describethe users of a social network, which may include, for example, propernames (first, middle and last of a person, a trade name and/or companyname of a business entity, etc.) biographic, demographic, and othertypes of descriptive information, such as work experience, educationalhistory, hobbies or preferences, geographic location, and additionaldescriptive data. By way of example, user profiles may include a user'sbirthday, relationship status, city of residence, and the like. Thesystem 20 may further store data describing one or more relationshipsbetween different users. The relationship information may indicate userswho have similar or common work experience, group memberships, hobbies,or educational history. A user profile may also include privacy settingsgoverning access to the user's information is to other users.

Client device 30 is generally a computer or computing device includingfunctionality for communicating (e.g., remotely) over a computernetwork. Client device 30 may be a desktop computer, laptop computer,personal digital assistant (PDA), in- or out-of-car navigation system,smart phone or other cellular or mobile phone, or mobile gaming device,among other suitable computing devices. Client device 30 may execute oneor more client applications, such as a web browser (e.g., MicrosoftWindows Internet Explorer, Mozilla Firefox, Apple Safari, Google Chrome,and Opera, etc.), to access and view content over a computer network. Inparticular implementations, the client applications allow a user ofclient device 30 to enter addresses of specific network resources to beretrieved, such as resources hosted by social networking system 20.These addresses can be Uniform Resource Locators, or URLs. In addition,once a page or other resource has been retrieved, the clientapplications may provide access to other pages or records when the user“clicks” on hyperlinks to other resources. By way of example, suchhyperlinks may be located within the web pages and provide an automatedway for the user to enter the URL of another page and to retrieve thatpage.

FIGS. 1 illustrates an example embodiment of a networking system,architecture, or infrastructure 100 (hereinafter referred to asnetworking system 100) that can implement the back end functions ofsocial networking system 20 illustrated in FIG. 3. In particularembodiments, networking system 100 enables users of networking system100 to interact with each other via social networking services providedby networking system 100 as well as with third parties. For example,users at remote user computing devices (e.g., personal computers,netbooks, multimedia devices, cellular phones (especially smart phones),etc.) may access networking system 100 via web browsers or other userclient applications to access websites, web pages, or web applicationshosted or accessible, at least in part, by networking system 100 to viewinformation, store or update information, communicate information, orotherwise interact with other users, third party websites, web pages, orweb applications, or other information stored, hosted, or accessible bynetworking system 100. In particular embodiments, networking system 100maintains a graph that includes graph nodes representing users,concepts, topics, and other information (data), as well as graph edgesthat connect or define relationships between graph nodes, as describedin more detail below.

With reference to FIGS. 1 and 5, in particular embodiments, networkingsystem 100 includes one or more data centers 102. For example,networking system 100 may include a plurality of data centers 102located strategically within various geographic regions for servingusers located within respective regions. In particular embodiments, eachdata center includes a number of client or web servers 104 (hereinafterclient servers 104) that communicate information to and from users ofnetworking system 100. For example, users at remote user computingdevices may communicate with client servers 104 via load balancers orother suitable systems via any suitable combination of networks andservice providers. Client servers 104 may query the caching systemdescribed herein in order to retrieve data to generate structureddocuments for responding to user requests.

Each of the client servers 104 communicates with one or more followerdistributed cache clusters or rings 106 (hereinafter follower cacheclusters 106). In the illustrated embodiment, data center 102 includesthree follower cache clusters 106 that each serve a subset of the webservers 104. In particular embodiments, a follower cache cluster 106 andthe client servers 104 the follower cache cluster 106 serves are locatedin close proximity, such as within a building, room, or othercentralized location, which reduces costs associated with theinfrastructure (e.g., wires or other communication lines, etc.) as wellas latency between the client servers 104 and respective servingfollower cache nodes cluster 106. However, in some embodiments, whileeach of the follower cache clusters 106, and the client servers 104 theyrespectively serve, may be located within a centralized location, eachof the follower cache clusters 106 and respective client servers 104 thefollower cache clusters 106 respectively serve, may be located in adifferent location than the other follower cache clusters 106 andrespective client servers 104 of a given data center; that is, thefollower cache clusters 106 (and the respective client servers 104 theclusters serve) of a given data center of a given region may bedistributed throughout various locations within the region.

In particular embodiments, each data center 102 further includes aleader cache cluster 108 that communicates information between thefollower cache clusters 106 of a given data center 102 and a persistentstorage database 110 of the given data center 102. In particularembodiments, database 110 is a relational database. In particularembodiments, leader cache cluster 108 may include a plug-in operative tointeroperate with any suitable implementation of database 110. Forexample, database 110 may be implemented as a dynamically-variableplug-in architecture and may utilize MySQL, and/or any suitablerelational database management system such as, for example, HAYSTACK,CASSANDRA, among others. In one implementation, the plug-in performsvarious translation operations, such as translating data stored in thecaching layer as graph nodes and edges to queries and commands suitablefor a relational database including one or more tables or flat files. Inparticular embodiments, leader cache cluster 108 also coordinates writerequests to database 110 from follower cache clusters 106 and sometimesread requests from follower cache clusters 106 for information cached inleader cache cluster 108 or (if not cached in leader cache cluster 108)stored in database 110. In particular embodiments, leader cache cluster108 further coordinates the synchronization of information stored in thefollower cache clusters 106 of the respective data center 102. That is,in particular embodiments, the leader cache cluster 108 of a given datacenter 102 is configured to maintain cache consistency (e.g., theinformation cached) between the follower cache clusters 106 of the datacenter 102, to maintain cache consistency between the follower cacheclusters 106 and the leader cache cluster 108, and to store theinformation cached in leader cache cluster 108 within database 110. Inone implementation, a leader cache cluster 108 and a follower cachecluster 106 can be considered a caching layer between client servers 104and database 110.

In one implementation, the caching layer is a write-thru/read-thrucaching layer, wherein all reads and writes traverse the caching layer.In one implementation, the caching layer maintains associationinformation and, thus, can handle queries for such information. Otherqueries are passed through to database 110 for execution. Database 110generally connotes a database system that may itself include othercaching layers for handling other query types.

Each follower cache cluster 106 may include a plurality of followercache nodes 112, each of which may be running on an individual computer,computing system, or server. However, as described above, each of thefollower cache nodes 112 of a given follower cache cluster 106 may belocated within a centralized location. Similarly, each leader cachecluster 108 may include a plurality of leader cache nodes 114, each ofwhich may be running on an individual computer, computing system, orserver. Similar to the follower cache nodes 112 of a given followercache cluster 106, each of the leader cache nodes 114 of a given leadercache cluster 108 may be located within a centralized location. Forexample, each data center 102 may include tens, hundreds, or thousandsof client servers 104 and each follower cache cluster 106 may includetens, hundreds, or thousands of follower cache nodes 112 that serve asubset of the client servers 104. Similarly, each leader cache cluster108 may include tens, hundreds, or thousands of leader cache nodes 114.In particular embodiments, each of the follower cache nodes 112 within agiven follower cache cluster 106 may only communicate with the otherfollower cache nodes 112 within the particular follower cache cluster106, the client servers 104 served by the particular follower cachecluster 106, and the leader cache nodes 114 within the leader cachecluster 108.

In particular embodiments, information stored by networking system 100is stored within each data center 102 both within database 110 as wellas within each of the follower and leader cache clusters 106 and 108,respectively. In particular embodiments, the information stored withineach database 110 is stored relationally (e.g., as objects and tablesvia MySQL), whereas the same information is stored within each of thefollower cache clusters 106 and the leader cache cluster 108 in a numberof data shards stored by each of the follower and leader cache clusters106 and 108, respectively, in the form of a graph including graph nodesand associations or connections between nodes (referred to herein asgraph edges). In particular embodiments, the data shards of each of thefollower cache clusters 106 and leader cache cluster 108 are bucketizedor divided among the cache nodes 112 or 114 within the respective cachecluster. That is, each of the cache nodes 112 or 114 within therespective cache cluster stores a subset of the shards stored by thecluster (and each set of shards stored by each of the follower andleader cache clusters 106 and 108, respectively, stores the sameinformation, as the leader cache cluster synchronizes the shards storedby each of the cache clusters of a given data center 102, and, in someembodiments, between data centers 102).

In particular embodiments, each graph node is assigned a uniqueidentifier (ID) (hereinafter referred to as node ID) that uniquelyidentifies the graph node in the graph stored by each of the followerand leader cache clusters 106 and 108, respectively, and database 110;that is, each node ID is globally unique. In one implementation, eachnode ID is a 64-bit identifier. In one implementation, a shard isallocated a segment of the node ID space. In particular embodiments,each node ID maps (e.g., arithmetically or via come mathematicalfunction) to a unique corresponding shard ID; that is, each shard ID isalso globally unique and refers to the same data object in each set ofshards stored by each of the follower and leader cache clusters 106 and108, respectively. In other words, all data objects are stored as graphnodes with unique node IDs and all the information stored in the graphin the data shards of each of the follower and leader cache clusters 106and 108, respectively, is stored in the data shards of each of thefollower and leader cache clusters 106 and 108, respectively, using thesame corresponding unique shard IDs.

As just described, in particular embodiments, the shard ID space (thecollection of shard IDs and associated information stored by all theshards of each cache cluster, and replicated in all of the otherfollower cache clusters 106 and leader cache cluster 108) is dividedamong the follower or leader cache nodes 112 and 114, respectively,within the follower or leader cache clusters 106 and 108, respectively.For example, each follower cache node 112 in a given follower cachecluster 106 may store a subset of the shards (e.g., tens, hundreds, orthousands of shards) stored by the respective follower cache cluster 106and each shard is assigned a range of node IDs for which to storeinformation, including information about the nodes whose respective nodeIDs map to the shard IDs in the range of shard IDs stored by theparticular shard. Similarly, each leader cache node 114 in the leadercache cluster 108 may store a subset of the shards (e.g., tens,hundreds, or thousands of shards) stored by the respective leader cachecluster 108 and each shard is assigned a range of node IDs for which tostore information, including information about the nodes whoserespective node IDs map to the shard IDs in the range of shard IDsstored by the particular shard.

However, as described above, a given shard ID corresponds to the samedata objects stored by the follower and leader cache clusters 106 and108, respectively. As the number of follower cache nodes 106 within eachfollower cache cluster 106 and the number of leader cache nodes 114within the leader cache cluster 108 may vary statically (e.g., thefollower cache clusters 106 and the leader cache cluster 108 maygenerally include different numbers of follower cache nodes 112 andleader cache nodes 114, respectively) or dynamically (e.g., cache nodeswithin a given cache cluster may be shut down for various reasonsperiodically or as needed for fixing, updating, or maintenance), thenumber of shards stored by each of the follower cache nodes 112 andleader cache nodes 114 may vary statically or dynamically within eachcache cluster as well as between cache clusters. Furthermore, the rangeof shard IDs assigned to each shard may also vary statically ordynamically.

In particular embodiments, each of the follower cache nodes 112 andleader cache nodes 114 includes graph management software that managesthe storing and serving of information cached within the respectivecache node. In particular embodiments, the graph management softwarerunning on each of the cache nodes of a given cache cluster maycommunicate to determine which shards (and corresponding shard IDs) arestored by each of the cache nodes within the respective cache cluster.Additionally, if the cache node is a follower cache node 112, the graphmanagement software running on the follower cache node 112 receivesrequests (e.g., write or read requests) from client servers 104, servesthe requests by retrieving, updating, deleting, or storing informationwithin the appropriate shard within the follower cache node, and managesor facilitates communication between the follower cache node 112 andother follower cache nodes 112 of the respective follower cache cluster106 as well as communication between the follower cache node 112 and theleader cache nodes 114 of the leader cache cluster 108. Similarly, ifthe cache node is a leader cache node 114, the graph management softwarerunning on the leader cache node 114 manages the communication betweenthe leader cache node 114 and follower cache nodes 112 of the followercache clusters 106 and the other leader cache nodes 114 of the leadercache cluster 108, as well as communication between the leader cachenode 114 and database 110. The graph management software running on eachof the cache nodes 112 and 114 understands that it is storing andserving information in the form of a graph.

In particular embodiments, the graph management software on eachfollower cache node 112 is also responsible for maintaining a table thatit shares with the other cache nodes 112 of the respective followercache cluster 106, the leader cache nodes 114 of the leader cachecluster 108, as well as the client servers 104 that the respectivefollower cache cluster 106 serves. This table provides a mapping of eachshard ID to the particular cache node 112 in a given follower cachecluster 106 that stores the shard ID and information associated with theshard ID. In this way, the client servers 104 served by a particularfollower cache cluster 106 know which of the follower cache nodes 112within the follower cache cluster 106 maintain the shard ID associatedwith information the client server 104 is trying to access, add, orupdate (e.g., a client server 104 may send write or read requests to theparticular follower cache node 112 that stores, or will store, theinformation associated with a particular shard ID after using themapping table to determine which of the follower cache nodes 112 isassigned, and stores, the shard ID). Similarly, in particularembodiments, the graph management software on each leader cache node 114is also responsible for maintaining a table that it shares with theother cache nodes 114 of the respective leader cache cluster 108, aswell as the follower cache nodes 112 of the follower cache clusters 106that the leader cache cluster 108 manages. Furthermore, in this way,each follower cache node 112 in a given follower cache cluster 106 knowswhich of the other follower cache nodes 112 in the given follower cachecluster 106 stores which shard IDs stored by the respective followercache cluster 106. Similarly, in this way each leader cache node 114 inthe leader cache cluster 108 knows which of the other leader cache nodes114 in the leader cache cluster 108 stores which shard IDs stored by theleader cache cluster 108. Furthermore, each follower cache node 112 in agiven follower cache cluster 106 knows which of the leader cache nodes114 in the leader cache cluster 108 stores which shard IDs. Similarly,each leader cache node 114 in the leader cache cluster 108 knows whichof the follower cache nodes 112 in each of the follower cache clusters106 stores which shard IDs.

In particular embodiments, information regarding each node in the graph,and in particular example embodiments a social graph, is stored in arespective shard of each of the follower cache clusters 106 and leadercache cluster 108 based on its shard ID. Each node in the graph, asdiscussed above, has a node ID. Along with the shard ID, the respectivecache node 112 or 114 may store a node type parameter identifying a typeof the node, as well as one or more name-value pairs (such as content(e.g., text, media, or URLs to media or other resources)) and metadata(e.g., a timestamp when the node was created or modified). In particularembodiments, each edge in the graph, and in particular exampleembodiments a social graph, is stored with each node the edge isconnected to. For example, most edges are bi-directional; that is, mostedges each connect two nodes in the graph. In particular embodiments,each edge is stored in the same shard with each node the edge connects.For example, an edge connecting node ID1 to node ID2 may be stored withthe shard ID corresponding to node ID1 (e.g., shard ID1) and with theshard ID corresponding to node ID2 (e.g., shard ID2), which may be indifferent shards or even different cache nodes of a given cache cluster.For example, the edge may be stored with shard ID1 in the form of {nodeID1, edge type, node ID2} where the edge type indicates the type ofedge. The edge may also include metadata (e.g., a timestamp indicatingwhen the edge was created or modified). The edge may also be cached withshard ID2 in the form of (node ID1, edge type, node ID2). For example,when a user of social networking system 100 establishes a contactrelationship with another user or a fan relationship with a concept oruser, the edge relationship of type “friend” or “fan” may be stored intwo shards, a first shared corresponding to the shard to which theuser's identifier is mapped and a second shard to which the objectidentifier of the other user or concept is mapped.

Networking system 100, and particularly the graph management softwarerunning on the follower cache nodes 112 of follower cache clusters 106and the leader cache nodes 114 of the leader cache cluster 108, supporta number of queries received from client servers 104 as well as to orfrom other follower or leader cache nodes 112 and 114, respectively. Forexample, the query object_add{ID1, node type1, metadata (not alwaysspecified), payload (not always specified)} causes the receiving cachenode to store a new node with the node ID1 specified in the query of thespecified node type1 in the shard the node ID1 corresponds to. Thereceiving cache node also stores with the node ID1 the metadata (e.g., atimestamp) and payload (e.g., name-value pairs and/or content such astext, media, resources, or references to resources), if specified. Asanother example, the query object_update{ID1, node type1 (not alwaysspecified), metadata (not always specified), payload (not alwaysspecified)} causes the receiving cache node to update the nodeidentified by node ID1 specified in the query (e.g., change the nodetype to the node type1 specified in the query, update the metadata withthe metadata specified in the query, or update the content stored withthe payload specified in the query) in the corresponding shard. Asanother example, the query object_delete{node ID1} causes the receivingcache node to delete the node identified by node ID1 specified in thequery. As another example, the query object_get{node ID1} causes thereceiving cache node to retrieve the content stored with the nodeidentified by node ID1 specified in the query.

Now referring to edge queries (as opposed to the node queries justdescribed), the query assoc_add{ID1, edge type1, ID2, metadata (notalways specified)} causes the receiving cache node (which stores nodeID1) to create an edge between the node identified by node ID1 and thenode identified by node ID2 of edge type edge type1 and to store theedge with the node identified by node ID1 along with the metadata (e.g.,a timestamp indicating when the edge was requested) if specified. Asanother example, the query assoc_update{node ID1, edge type1, node ID2,metadata (not always specified)} causes the receiving cache node (whichstores node ID1) to update the edge between the node identified by nodeID1 and the node identified by node ID2. As another example, the queryassoc_delete{node ID1, edge type1 (not always specified), node ID2}causes the receiving cache node (which stores node ID1) to delete theedge between the node identified by node ID1 and the node identified bynode ID2. As another example, the query assoc_get{node ID1, edge type1,sortkey (not always specified), start (not always specified), limit (notalways specified)} causes the receiving cache node (which stores nodeID1) to return the node IDs of the nodes connected to the nodeidentified by node ID1 by edges of edge type1. Additionally, ifspecified, the sortkey specifies a filter. For example, if the sortkeyspecifies a timestamp, the receiving cache node (which stores node ID1)returns the node IDs of the nodes connected to the node identified bynode ID1 by edges of edge type1 which were created between the timevalue specified by the start parameter and the time value specified bythe limit parameter. As another example, the query assoc_exists{nodeID1, edge type1, list of other node IDs, sortkey (not always specified),start (not always specified), limit (not always specified)} causes thereceiving cache node (which stores node ID1) to return the node IDs ofthe nodes specified in the list of other node IDs connected to the nodeidentified by shard ID1 by edges of edge type1. In addition, the queriesdescribed above may be sent in the described form and used to update theleader cache nodes 114.

In one implementation, the caching layer implemented by the follower andleader cache clusters 108 and 106 cache maintain association data in oneor more indexes in a manner that supports high query rates for one ormore query types. In some implementations, the invention facilitatesefficient intersection, membership and filtering queries directed toassociations between nodes in the graph. For example, in oneimplementation, the caching layer caches information in a manneroptimized to handle point lookup, range and count queries for a varietyof associations between nodes. For example, in constructing a page, aclient server 104 may issue a query for all friends of a given user. Theclient server 104 may issue an assoc_get query identifying the user andthe “friend” edge type. To facilitate handling of the query, a cachenode in the caching layer may store associations of a given type (suchas “friends”, “fans”, “members”, “likes”, etc.) between a first node(e.g., a node corresponding to a user) and a node corresponding tocontacts or friends of a user. In addition, to construct another partyof the page, a client server 104 may issue a query of the last N set ofwall posts on the profile, by issuing a assoc_get query identifying theuser or user profile, the “wallpost” edge type and a limit value.Similarly, comments to a particular wall post can be retrieved in asimilar manner.

In one implementation, the caching layer implemented by the followercache clusters 106 and the leader cache clusters maintain a set ofin-memory structures for associations between nodes (id1, id2) in thegraph that facilitate fast searching and handle high query rates. Forexample, for each (id1,type) association set (a set of all associationsthat originate at id1 and have a given type), the caching layermaintains two in-memory indexes. As discussed above, these associationsets are maintained by cache nodes in each cluster that based on theshard in which id1 falls. Still further, given the structure discussedbelow, a given association between two nodes may be stored in twoassociation sets each directed to the respective nodes of theassociation. A first index is based on a temporal attribute (e.g., timestamps) and supports range queries. A second index by id2 does notsupport range queries, but supports better time complexity of insertsand look ups. In one implementation, the first index is an ordereddynamic array of association entries stored in a circular buffer. Eachentry in the circular buffer describes or corresponds to one associationand contains the following fields: a) $flags (1 byte) (indicating thevisibility of an association); b) $id2 (8 bytes); c) $time (4 bytes); d)$data (8 bytes) ($data is a fixed size 8 byte field (when more than 8bytes are needed for $data, this becomes a pointer to another memorychunk to hold the full $data value; $data is optional for a given assoctype); and e) $link (8 bytes) offsets of next and previous entries inthe same id2 index bucket (see below). In one implementation, the arrayis ordered by the $time attribute ascending. The number of entries inthe index is capped (such as 10,000) and configurable by associationtype. When the limit is reached the array wraps around. Because thearray is $time-sorted, most new entries will be appended at the endwithout shifting any of the existing elements.

In one implementation, the primary index can be stored in a singlememcache key that can be looked up by name (“assoc:<id1>:<type>”)through a global memcached hash table. The array can be fronted with aheader containing the following fields: a) count (4 bytes): the count ofvisible associations in the (id1,type) association set (storedpersistently, not just the cached entries in the index); b) head (4bytes): the byte offset of array head (element that sorts highest) inthe circular buffer; c) tail (4 bytes): the byte offset of array tail(element that sorts lowest) in the circular buffer; and d) id2 indexpointer (8 bytes): a pointer to a block containing an id2 hash table.

The second ($id2) index is implemented, in one embodiment, as a hashtable and supports quick inserts and lookups for a given($id1,$type,$id2) association. The hash table itself, in oneimplementation, may be stored in a separate block allocated withmemcached's memory allocator. The table is an array of offsets into theprimary index, each identifying the first element in the correspondinghash bucket. Elements are linked into a bucket through their $linkfields. Storing the hash table in a separate block allows implementersto resize the table and the primary index independently, thus reducingthe amount of memory copied as the association set grows. Linkingassociation entries into buckets in-place also improves memoryefficiency. The hash table (and bucket lists) may need to be rebuiltwhen entries marked hidden or deleted are expunged from the index, butthis can be done infrequently.

Accordingly, as a new association of the same <type> is added, a cachenode 112, 114 ads the newly associated object to the hash table and thecircular buffer, removing the oldest entry from the circular buffer. Asdiscussed above, the <sortkey> value can be used to sort matchingentries based on the attribute, such as a time stamps. In addition, a<limit> value limits the number of returned results to the first Nvalues, where N=<limit>. This configuration allows for serving queriesregarding associations between nodes at a very high query rate. Forexample, a first query may ask to display a set of friends in a sectionof a web page. A cache node can quickly respond to a get_assoc (id1,type, sortkey, limit) query by looking up association set correspondingto id1 by accessing the primary index and retrieving the first N (whereN=limit) id2 entries in the circular buffer. In addition, the hash tableof the secondary index facilitates point look ups. Still further, thecount value maintained by the caching layer facilitates fast responsesto the count of a given association set (id1, type).

Some general examples of storing and serving data will now be described(more specific examples relating to particular example implementationsof a social graph will be described later after the particular exampleimplementations of the social graph are described). For example, when aclient server 104 receives a request for a web page, such as from a userof networking system 100, or from another server, component,application, or process of networking system 100 (e.g., in response to auser request), the client server 104 may need to issue one or morequeries in order to generate the requested web page. In addition, as auser interacts with networking system 100, the client server 104 mayreceive requests that establish or modify object nodes and/orassociations be object nodes. In some instances, the request received bya client server 104 generally includes the node ID representing the useron whose behalf the request to the client server 104 was made. Therequest may also, or alternately, include one or more other node IDscorresponding to objects the user may want to view, update, delete, orconnect or associate (with an edge).

For example, a request may be a read request for accessing informationassociated with the object or objects the user wants to view (e.g., oneor more objects for serving a web page). For example, the read requestmay be a request for content stored for a particular node. For example,a wall post on a user profile can be represented as a node with an edgetype of “wallpost.” Comments to the wallpost can also be represented asnodes in the graph with edge type “comment” associations to thewallpost. In such an example, in particular embodiments, the clientserver 104 determines the shard ID corresponding to the node ID of theobject (node) that includes the content or other information requested,uses the mapping table to determine which of the follower cache nodes112 (in the follower cache cluster 106 that serves the client server104) stores the shard ID, and transmits a query including the shard IDto the particular one of the follower cache nodes 112 storing theinformation associated with and stored with the shard ID. The particularcache node 112 then retrieves the requested information (if cachedwithin the corresponding shard) and transmits the information to therequesting client server 104, which may then serve the information tothe requesting user (e.g., in the form of an HTML or other structureddocument that is renderable by the web browser or otherdocument-rendering application running on the user's computing device.If the requested information is not stored/cached within the followercache node 112, the follower cache node 112 may then determine, usingthe mapping table, which of the leader cache nodes 114 stores the shardstoring the shard ID and forwards the query to the particular leadercache node 114 that stores the shard ID. If the requested information iscached within the particular leader cache node 114, the leader cachenode 114 may then retrieve the requested information and forward it tothe follower cache node 112, which then updates the particular shard inthe follower cache node 112 to store the requested information with theshard ID and proceeds to serve the query as just described to the clientserver 104, which may then serve the information to the requesting user.If the requested information is not cached within the leader cache node114, the leader cache node 114 may then translate the query into thelanguage of database 110, and transmit the new query to database 110,which then retrieves the requested information and transmits therequested information to the particular leader cache node 114. Theleader cache node 114 may then translate the retrieved information backinto the graphical language understood by the graph management software,update the particular shard in the leader cache node 114 to store therequested information with the shard ID, and transmit the retrievedinformation to the particular follower cache node 112, which thenupdates the particular shard in the follower cache node 112 to store therequested information with the shard ID and proceeds to serve the queryas just described to the client server 104, which may then serve theinformation to the requesting user.

As another example, the user request may be a write request to updateexisting information or store additional information for a node or tocreate or modify an edge between two nodes. In the former case, if theinformation to be stored is for a non-existing node, the client server104 receiving the user request transmits a request for a node ID for anew node to the respective follower cache cluster 106 serving the clientserver 104. In some cases or embodiments, the client server 104 mayspecify a particular shard within which the new node is to be stored(e.g., to co-locate the new node with another node). In such a case, theclient server 104 requests a new node ID from the particular followercache node 112 storing the specified shard. Alternately, the clientserver 104 may pass a node ID of an existing node with the request for anew node ID to the follower cache node 112 storing the shard that storesthe passed node ID to cause the follower cache node 112 to respond tothe client server 104 with a node ID for the new node that is in therange of node IDs stored in the shard. In other cases or embodiments,the client server 104 may select (e.g., randomly or based on somefunction) a particular follower cache node 112 or a particular shard tosend the new node ID request to. Whatever the case, the particular cachenode 112, or more particularly the graph management software running onthe follower cache node 112, then transmits the new node ID to theclient server 104. The client server 104 may then formulate a writerequest that includes the new node ID to the corresponding followercache node 112. The write request may also specify a node type of thenew node and include a payload (e.g., content to be stored with the newnode) and/or metadata (e.g., the node ID of the user making the request,a timestamp indicating when the request was received by the clientserver 104, among other data) to be stored with the node ID. Forexample, the write request sent to the follower cache node 112 may be ofthe form object_add{node ID, node type, payload, metadata}. Similarly,to update a node, the client server 104 may send a write request of theform object_modify{node ID, node type, payload, metadata} to thefollower cache node 112 storing the shard within which the node ID isstored. Similarly, to delete a node, the client server 104 may send arequest of the form object_delete{node ID} to the follower cache node112 storing the shard within which the shard ID is stored.

In particular embodiments, the follower cache node then transmits therequest to the leader cache node 114 storing the shard that stores thecorresponding node ID so that the leader cache node 114 may then updatethe shard. The leader cache node 114 then translates the request intothe language of database 110 and transmits the translated request to thedatabase 110 so that the database may then be updated.

FIG. 4 illustrates an example method for processing a request to add anassociation (assoc_add) between two nodes. As FIG. 4 illustrates, when afollower cache node 112 receives an assoc_add request (e.g.,assoc_add(id1, type, id2, metadata), it accesses an index to identifythe association set object corresponding to id1 and type (402). Followercache nodes 112 adds id2 to both the hash table and the circular bufferof the association set object and increments the count value of theassociation set object (404). The association set object now maintainsthe new association of the given type between node id1 and node id2. Tofacilitate searching of the association relative to id2, follower cachenode 112 identifies the shard Id corresponding to the node identifierid2 and forwards the assoc_add request to the follower cache node 112 inthe cluster that handles the identified shard (406). If the instantfollower cache node 112 handles the shard, it processes the assoc_addrequest. In one implementation, the forwarding follower cache node 112may transmit a modified assoc_add request that signals that this is anupdate required to establish a bi-directional association in the cachelayer. The follower cache node 112 also forwards the assoc_add requestto the leader cache node 114 corresponding to the shard in which id1falls (408). The leader cache node 114 may execute a similar process toestablish a bi-directional association in the leader cache cluster. Theleader cache node 114 also causes the new association to be persisted indatabase 110. In this manner, an association between node id1 and nodeid2 is now searchable in an index with reference to id1 and type, andseparately, id2 and type.

In particular embodiments, the graph can maintain a variety of differentnode types, such as users, pages, events, wall posts, comments,photographs, videos, background information, concepts, interests and anyother element that would be useful to represent as a node. Edge typescorrespond to associations between the nodes and can include friends,followers, subscribers, fans, likes (or other indications of interest),wallpost, comment, links, suggestions, recommendations, and other typesof associations between nodes. In one implementation, a portion of thegraph can be a social graph including user nodes that each correspond toa respective user of the social network environment. The social graphmay also include other nodes such as concept nodes each devoted ordirected to a particular concept as well as topic nodes, which may ormay not be ephemeral, each devoted or directed to a particular topic ofcurrent interest among users of the social network environment. Inparticular embodiments, each node has, represents, or is represented by,a corresponding web page (“profile page”) hosted or accessible in thesocial network environment. By way of example, a user node may have acorresponding user profile page in which the corresponding user can addcontent, make declarations, and otherwise express himself or herself. Byway of example, as will be described below, various web pages hosted oraccessible in the social network environment such as, for example, userprofile pages, concept profile pages, or topic profile pages, enableusers to post content, post status updates, post messages, post commentsincluding comments on other posts submitted by the user or other users,declare interests, declare a “like” (described below) towards any of theaforementioned posts as well as pages and specific content, or tootherwise express themselves or perform various actions (hereinafterthese and other user actions may be collectively referred to as “posts”or “user actions”). In some embodiments, posting may include linking to,or otherwise referencing additional content, such as media content(e.g., photos, videos, music, text, etc.), uniform resource locators(URLs), and other nodes, via their respective profile pages, other userprofile pages, concept profile pages, topic pages, or other web pages orweb applications. Such posts, declarations, or actions may then beviewable by the authoring user as well as other users. In particularembodiments, the social graph further includes a plurality of edges thateach define or represent a connection between a corresponding pair ofnodes in the social graph. As discussed above, each item of content maybe a node in the graph linked to other nodes.

As just described, in various example embodiments, one or more describedweb pages or web applications are associated with a social networkenvironment or social networking service. As used herein, a “user” maybe an individual (human user), an entity (e.g., an enterprise, business,or third party application), or a group (e.g., of individuals orentities) that interacts or communicates with or over such a socialnetwork environment. As used herein, a “registered user” refers to auser that has officially registered within the social networkenvironment (Generally, the users and user nodes described herein referto registered users only, although this is not necessarily a requirementin other embodiments; that is, in other embodiments, the users and usernodes described herein may refer to users that have not registered withthe social network environment described herein). In particularembodiments, each user has a corresponding “profile” page stored,hosted, or accessible by the social network environment and viewable byall or a selected subset of other users. Generally, a user hasadministrative rights to all or a portion of his or her own respectiveprofile page as well as, potentially, to other pages created by or forthe particular user including, for example, home pages, pages hostingweb applications, among other possibilities. As used herein, an“authenticated user” refers to a user who has been authenticated by thesocial network environment as being the user claimed in a correspondingprofile page to which the user has administrative rights or,alternately, a suitable trusted representative of the claimed user.

A connection between two users or concepts may represent a definedrelationship between users or concepts of the social networkenvironment, and can be defined logically in a suitable data structureof the social network environment as an edge between the nodescorresponding to the users, concepts, events, or other nodes of thesocial network environment for which the association has been made. Asused herein, a “friendship” represents an association, such as a definedsocial relationship, between a pair of users of the social networkenvironment. A “friend,” as used herein, may refer to any user of thesocial network environment with which another user has formed aconnection, friendship, association, or relationship with, causing anedge to be generated between the two users. By way of example, tworegistered users may become friends with one another explicitly such as,for example, by one of the two users selecting the other for friendshipas a result of transmitting, or causing to be transmitted, a friendshiprequest to the other user, who may then accept or deny the request.Alternately, friendships or other connections may be automaticallyestablished. Such a social friendship may be visible to other users,especially those who themselves are friends with one or both of theregistered users. A friend of a registered user may also have increasedaccess privileges to content, especially user-generated or declaredcontent, on the registered user's profile or other page. It should benoted, however, that two users who have a friend connection establishedbetween them in the social graph may not necessarily be friends (in theconventional sense) in real life (outside the social networkingenvironment). For example, in some implementations, a user may be abusiness or other non-human entity, and thus, incapable of being afriend with a human being user in the traditional sense of the word.

As used herein, a “fan” may refer to a user that is a supporter orfollower of a particular user, web page, web application, or other webcontent accessible in the social network environment. In particularembodiments, when a user is a fan of a particular web page (“fans” theparticular web page), the user may be listed on that page as a fan forother registered users or the public in general to see. Additionally, anavatar or profile picture of the user may be shown on the page (or in/onany of the pages described below). As used herein, a “like” may refer tosomething, such as, by way of example and not by way of limitation, apost, a comment, an interest, a link, a piece of media (e.g., photo,photo album, video, song, etc.) a concept, an entity, or a page, amongother possibilities (in some implementations a user may indicate ordeclare a like to or for virtually anything on any page hosted by oraccessible by the social network system or environment), that a user,and particularly a registered or authenticated user, has declared orotherwise demonstrated that he or she likes, is a fan of, supports,enjoys, or otherwise has a positive view of In one embodiment, toindicate or declare a “like” or to indicate or declare that the user isa “fan” of something may be processed and defined equivalently in thesocial networking environment and may be used interchangeably;similarly, to declare oneself a “fan” of something, such as a concept orconcept profile page, or to declare that oneself “likes” the thing, maybe defined equivalently in the social networking environment and usedinterchangeably herein. Additionally, as used herein, an “interest” mayrefer to a user-declared interest, such as a user-declared interestpresented in the user's profile page. As used herein, a “want” may referto virtually anything that a user wants. As described above, a “concept”may refer to virtually anything that a user may declare or otherwisedemonstrate an interest in, a like towards, or a relationship with, suchas, by way of example, a sport, a sports team, a genre of music, amusical composer, a hobby, a business (enterprise), an entity, a group,a celebrity, a person who is not a registered user, or even, an event,in some embodiments, another user (e.g., a non-authenticated user), etc.By way of example, there may be a concept node and concept profile pagefor “Jerry Rice,” the famed professional football player, created andadministered by one or more of a plurality of users (e.g., other thanJerry Rice), while the social graph additionally includes a user nodeand user profile page for Jerry Rice created by and administered byJerry Rice, himself (or trusted or authorized representatives of JerryRice).

FIG. 5 illustrates a distributed, redundant system. In theimplementation shown, the distributed redundant system includes at leastfirst and second data centers 102 a, 102 b. Each of the data centers 102a, 102 b includes one or more follower cache clusters 106 and a leadercache cluster 108 a, 108 b. In one implementation, leader cache cluster108 a acts as a primary (master) cache cluster, while leader cachecluster 108 b is a secondary (slave) cache cluster. In oneimplementation, data centers 102 a, 102 b are redundant in the sensethat synchronization functions are employed to achieve replicated copiesof the database 110. In one implementation, data center 102 a may bephysically located at one geographic region (such as the West Coast ofthe United States) to serve traffic from that region, while data center102 b may be physically located at another geographic region (such asthe East Coast of the United States). Given that users from either ofthese regions may access the same data and associations, efficientsynchronization mechanisms are desired.

FIG. 6 illustrates an example method of how a leader cache node 114processes write commands. As discussed above and with reference to FIG.5, a follower cache node 112 may receive a write command to add/updatean object or association from a client server 104 (FIG. 5, No. 1). Thefollower cache node 112 forwards the write command to a correspondingleader cache node 114 (FIG. 5, No. 2). When the leader cache node 114receives a write command from a follower cache node (602), it processesthe write command to update one or more entries in the cache maintainedby the leader cache cluster 108 a (604) and writes the update topersistent database 110 a (606) (FIG. 5, No. 3). The leader cache node114 also acknowledges the write command (ACK) to the follower cache node112 and broadcasts the update to other follower cache clusters 106 ofthe data center 102 a (FIG. 5, No. 4 a) and the secondary leader cachecluster 108 b, which forwards the update to its follower cache clusters106 (FIG. 5, No. 4 b) (608). As FIG. 6 illustrates, the leader cachenode 114 also adds the update to a replication log (610). The databases110 a, 110 b implement a synchronization mechanism, such as MySQLReplication, to synchronize the persistent databases.

FIG. 7 illustrates a message flow according to one implementation of theinvention. When a write command is received at a follower cache node 112in a ring 106 that is not directly associated with the primary leadercache cluster 108 a (FIG. 7, No. 1), the follower cache node 112forwards the write message to the primary leader cache cluster 108 a forprocessing (FIG. 7, No. 2). A leader cache node 114 in the primaryleader cache cluster 108 a may then broadcast the update to its followercache clusters 106 (FIG. 7, No. 3) and writes the changes to database110 a. As FIG. 7 shows, the follower cache node 112 that received thewrite command may also forward the write command to its secondary leadercache cluster 108 b (FIG. 7, No. 5), which broadcasts the updates toother follower cache clusters 106 (FIG. 7, No. 5). The foregoingarchitecture allows for therefore allows for changes to the cachinglayer to be quickly replicated across data centers, while the separatereplication between databases 110 a, 110 b allow for data security.

The applications or processes described herein can be implemented as aseries of computer-readable instructions, embodied or encoded on orwithin a tangible data storage medium, that when executed are operableto cause one or more processors to implement the operations describedabove. While the foregoing processes and mechanisms can be implementedby a wide variety of physical systems and in a wide variety of networkand computing environments, the computing systems described belowprovide example computing system architectures of the server and clientsystems described above, for didactic, rather than limiting, purposes.

FIG. 2 illustrates an example computing system architecture, which maybe used to implement a server 22 a, 22 b. In one embodiment, hardwaresystem 1000 comprises a processor 1002, a cache memory 1004, and one ormore executable modules and drivers, stored on a tangible computerreadable medium, directed to the functions described herein.Additionally, hardware system 1000 includes a high performanceinput/output (I/O) bus 1006 and a standard I/O bus 1008. A host bridge1010 couples processor 1002 to high performance I/O bus 1006, whereasI/O bus bridge 1012 couples the two buses 1006 and 1008 to each other. Asystem memory 1014 and one or more network/communication interfaces 1016couple to bus 1006. Hardware system 1000 may further include videomemory (not shown) and a display device coupled to the video memory.Mass storage 1018, and I/O ports 1020 couple to bus 1008. Hardwaresystem 1000 may optionally include a keyboard and pointing device, and adisplay device (not shown) coupled to bus 1008. Collectively, theseelements are intended to represent a broad category of computer hardwaresystems, including but not limited to general purpose computer systemsbased on the x86-compatible processors manufactured by Intel Corporationof Santa Clara, Calif., and the x86-compatible processors manufacturedby Advanced Micro Devices (AMD), Inc., of Sunnyvale, Calif., as well asany other suitable processor.

The elements of hardware system 1000 are described in greater detailbelow. In particular, network interface 1016 provides communicationbetween hardware system 1000 and any of a wide range of networks, suchas an Ethernet (e.g., IEEE 802.3) network, a backplane, etc. Massstorage 1018 provides permanent storage for the data and programminginstructions to perform the above-described functions implemented in theservers 22 a, 22 b, whereas system memory 1014 (e.g., DRAM) providestemporary storage for the data and programming instructions whenexecuted by processor 1002. I/O ports 620 are one or more serial and/orparallel communication ports that provide communication betweenadditional peripheral devices, which may be coupled to hardware system1000.

Hardware system 1000 may include a variety of system architectures; andvarious components of hardware system 1000 may be rearranged. Forexample, cache 1004 may be on-chip with processor 1002. Alternatively,cache 1004 and processor 1002 may be packed together as a “processormodule,” with processor 1002 being referred to as the “processor core.”Furthermore, certain embodiments of the present invention may notrequire nor include all of the above components. For example, theperipheral devices shown coupled to standard I/O bus 1008 may couple tohigh performance I/O bus 1006. In addition, in some embodiments, only asingle bus may exist, with the components of hardware system 1000 beingcoupled to the single bus. Furthermore, hardware system 1000 may includeadditional components, such as additional processors, storage devices,or memories.

In one implementation, the operations of the embodiments describedherein are implemented as a series of executable modules run by hardwaresystem 1000, individually or collectively in a distributed computingenvironment. In a particular embodiment, a set of software modulesand/or drivers implements a network communications protocol stack,browsing and other computing functions, optimization processes, and thelike. The foregoing functional modules may be realized by hardware,executable modules stored on a computer readable medium, or acombination of both. For example, the functional modules may comprise aplurality or series of instructions to be executed by a processor in ahardware system, such as processor 1002. Initially, the series ofinstructions may be stored on a storage device, such as mass storage1018. However, the series of instructions can be tangibly stored on anysuitable storage medium, such as a diskette, CD-ROM, ROM, EEPROM, etc.Furthermore, the series of instructions need not be stored locally, andcould be received from a remote storage device, such as a server on anetwork, via network/communications interface 1016. The instructions arecopied from the storage device, such as mass storage 1018, into memory1014 and then accessed and executed by processor 1002.

An operating system manages and controls the operation of hardwaresystem 1000, including the input and output of data to and from softwareapplications (not shown). The operating system provides an interfacebetween the software applications being executed on the system and thehardware components of the system. Any suitable operating system may beused, such as the LINUX Operating System, the Apple Macintosh OperatingSystem, available from Apple Computer Inc. of Cupertino, Calif., UNIXoperating systems, Microsoft (r) Windows(r) operating systems, BSDoperating systems, and the like. Of course, other implementations arepossible. For example, the nickname generating functions describedherein may be implemented in firmware or on an application specificintegrated circuit.

Furthermore, the above-described elements and operations can becomprised of instructions that are stored on storage media. Theinstructions can be retrieved and executed by a processing system. Someexamples of instructions are software, program code, and firmware. Someexamples of storage media are memory devices, tape, disks, integratedcircuits, and servers. The instructions are operational when executed bythe processing system to direct the processing system to operate inaccord with the invention. The term “processing system” refers to asingle processing device or a group of inter-operational processingdevices. Some examples of processing devices are integrated circuits andlogic circuitry. Those skilled in the art are familiar withinstructions, computers, and storage media.

The present disclosure encompasses all changes, substitutions,variations, alterations, and modifications to the example embodimentsherein that a person having ordinary skill in the art would comprehend.Similarly, where appropriate, the appended claims encompass all changes,substitutions, variations, alterations, and modifications to the exampleembodiments herein that a person having ordinary skill in the art wouldcomprehend. By way of example, while embodiments of the presentinvention have been described as operating in connection with a socialnetworking website, the present invention can be used in connection withany communications facility that supports web applications and modelsdata as a graph of associations. Furthermore, in some embodiments theterm “web service” and “web-site” may be used interchangeably andadditionally may refer to a custom or generalized API on a device, suchas a mobile device (e.g., cellular phone, smart phone, personal GPS,personal digital assistance, personal gaming device, etc.), that makesAPI calls directly to a server.

What is claimed is:
 1. A system comprising: a database; and a cachelayer comprising one or more cache nodes, the one or more cache nodesoperative to: maintain in a memory one or more data structures storingassociation information describing associations between nodes in a grapha plurality of distributed cache clusters for storing information in theform of a graph, the graph comprising a plurality of nodes, eachuniquely identified by a node identifier, and edge informationindicating associations between nodes; respond to queries forassociations between nodes in the graph by accessing the memory; andforward other queries to the database for processing.
 2. The system ofclaim 1 wherein the one or more cache nodes are operative to: receive arequest to change the association information in the memory; change theassociation in the memory; modify the request for implementation by thedatabase; and forward the modified request to the database forprocessing.
 3. The system of claim 1 wherein the cache layer comprises aplurality of distributed cache clusters, each cache cluster comprising aplurality of cache nodes, each cache cluster allocated one or more datashards, each of the cache nodes of a cache cluster storing associationinformation for nodes corresponding to the allocated data shards.
 4. Thesystem of claim 3 wherein the plurality of cache clusters comprise: aleader cache cluster comprising a plurality of leader nodes; and one ormore follower cache clusters each comprising a plurality of followercache nodes.
 5. The system of claim 1 wherein the one or more cachenodes are operative to: maintain in a memory, for each association setcorresponding to a first node of a plurality of nodes and an associationtype of a plurality of association types, a first index and a secondindex; wherein the first index comprises an ordered array of entries,each entry including a node identifier of a second node that isassociated with the first node and a sorting attribute; wherein thesecond index comprises a hash table comprising entries corresponding tothe node identifiers of respective second nodes that are associated withthe first node; receive a command to add an association of a firstassociation type between a first node and a second node, the commandincluding a first node identifier and a second node identifier; andaccess the memory against the first association type and the first nodeidentifier to add the second node identifier to a first index and asecond index corresponding to the first association type and the firstnode identifier.
 6. The system of claim 5 wherein the one or more cachenodes are operative to: maintain a count value for each association set;increment count values in response to commands to add an associationcorresponding to respective association sets; and decrement count valuesin response to commands to delete an association corresponding torespective association sets.
 7. A method comprising, by a cache layercomprising one or more cache nodes, each cache node: maintaining in amemory one or more data structures storing association informationdescribing associations between nodes in a graph a plurality ofdistributed cache clusters for storing information in the form of agraph, the graph comprising a plurality of nodes, each uniquelyidentified by a node identifier, and edge information indicatingassociations between nodes; responding to queries for associationsbetween nodes in the graph by accessing the memory; and forwarding otherqueries to a database for processing.
 8. The method of claim 7 furthercomprising, by each cache node: receiving a request to change theassociation information in the memory; changing the association in thememory; and modifying the request for implementation by the database;and forwarding the modified request to the database for processing. 9.The method of claim 7 wherein the cache layer comprises a plurality ofdistributed cache clusters, each cache cluster comprising a plurality ofcache nodes, each cache cluster allocated one or more data shards, eachof the cache nodes of a cache cluster storing association informationfor nodes corresponding to the allocated data shards.
 10. The method ofclaim 9 wherein the plurality of cache clusters comprise: a leader cachecluster comprising a plurality of leader nodes; and one or more followercache clusters each comprising a plurality of follower cache nodes. 11.The method of claim 7 further comprising, by each cache node:maintaining in a memory, for each association set corresponding to afirst node of a plurality of nodes and an association type of aplurality of association types, a first index and a second index;wherein the first index comprises an ordered array of entries, eachentry including a node identifier of a second node that is associatedwith the first node and a sorting attribute; wherein the second indexcomprises a hash table comprising entries corresponding to the nodeidentifiers of respective second nodes that are associated with thefirst node; receiving a command to add an association of a firstassociation type between a first node and a second node, the commandincluding a first node identifier and a second node identifier; andaccessing the memory against the first association type and the firstnode identifier to add the second node identifier to a first index and asecond index corresponding to the first association type and the firstnode identifier.
 12. The method of claim 11 further comprising, by eachcache node: maintaining a count value for each association set;incrementing count values in response to commands to add an associationcorresponding to respective association sets; and decrementing countvalues in response to commands to delete an association corresponding torespective association sets.
 13. A non-transitory storage medium storingcomputer-readable instructions, the instructions, when executed,operative to cause one or more processors to operate as a cache node ina cache layer comprising one or more cache nodes, the one or more cachenodes operative to: maintain in a memory one or more data structuresstoring association information describing associations between nodes ina graph a plurality of distributed cache clusters for storinginformation in the form of a graph, the graph comprising a plurality ofnodes, each uniquely identified by a node identifier, and edgeinformation indicating associations between nodes; respond to queriesfor associations between nodes in the graph by accessing the memory; andforward other queries to the database for processing.
 14. The storagemedium of claim 13 wherein the one or more cache nodes are operative to:receive a request to change the association information in the memory;change the association in the memory; modify the request forimplementation by the database; and forward the modified request to thedatabase for processing.
 15. The storage medium of claim 13 wherein thecache layer comprises a plurality of distributed cache clusters, eachcache cluster comprising a plurality of cache nodes, each cache clusterallocated one or more data shards, each of the cache nodes of a cachecluster storing association information for nodes corresponding to theallocated data shards.
 16. The storage medium of claim 13 wherein theplurality of cache clusters comprise: a leader cache cluster comprisinga plurality of leader nodes; and one or more follower cache clusterseach comprising a plurality of follower cache nodes.
 17. The storagemedium of claim 13 wherein the one or more cache nodes are operative to:maintain in a memory, for each association set corresponding to a firstnode of a plurality of nodes and an association type of a plurality ofassociation types, a first index and a second index; wherein the firstindex comprises an ordered array of entries, each entry including a nodeidentifier of a second node that is associated with the first node and asorting attribute; wherein the second index comprises a hash tablecomprising entries corresponding to the node identifiers of respectivesecond nodes that are associated with the first node; receive a commandto add an association of a first association type between a first nodeand a second node, the command including a first node identifier and asecond node identifier; and access the memory against the firstassociation type and the first node identifier to add the second nodeidentifier to a first index and a second index corresponding to thefirst association type and the first node identifier.
 18. The storagemedium of claim 13 wherein the one or more cache nodes are operative to:maintain a count value for each association set; increment count valuesin response to commands to add an association corresponding torespective association sets; and decrement count values in response tocommands to delete an association corresponding to respectiveassociation sets.